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1. Introduction 

In the present paper, the discounted stochastic optimal control problem is dealt with. This kind of 
problem is very popular and plentifully used in many domains, especially in stochastic finance since 
it leads to maximizing the average discounted agent's utility. The method of solving the problem used 
here is the maximum principle which, in deterministic setting, was formulated in 1950s by the group 
of L. S. Pontryagin. For diffusions, the maximum principle has been studied by many researchers. 
The earliest versions of a maximum principle for such process were suggested by Kushner j 14| and 
Bismut |5|. Further progress on the subject was subsequently made by Bensoussan |4|, Peng 1221 . 
Cadenillas and Haussmann |[7jl and others. Originally, the main technical tool used when considering 
maximum principle was the calculus of variations which was not easy to apply to real examples and 
was not convenient for numerical simulations. The turning point which led to its further intensive 
study was the paper by Pardoux and Peng (21 1 where the general (nonlinear) problem of Backward 
Stochastic Differential Equation (BSDE in short) was formulated and the existence and uniqueness 
theorems were given. BSDE theory provides an elegant and easy-to-handle tool to describe the ad- 
joint processes and to formulate the maximum principle by means of the Hamiltonian function. Also, 
numerical methods for this kind of processes are intensively studied. 

These results were extended in numerous papers. For example, in case of diffusions with jumps, a 
necessary maximum principle on the finite time horizon was formulated by Tang and Li |25 1 whereas 
sufficient optimality conditions on finite time horizon were specified by 0ksendal, Sulem and Fram- 
stad [171. Maximum principle on infinite time horizon was studied in Haadem, Proske and 0ksendal 
II2I . Interested reader can also find a large amount of papers on maximum principle for a variety 
of cases: Singular control (Bahlali and Mezerdi ||3], Dufour and Miller |9|, 0ksendal and Sulem 
fT6l ), Impulse control (Chikodza |8], Wu and Zhang fWi), Controlled SPDE's (Al-Hussein |2l, Fu- 
hrman, Hu and Tessitore 11 II ). Delayed controlled systems (Agram, Haadem, 0ksendal and Proske 
m . 0ksendal, Sulem and Zhang II8I ). Near-optimal control (Zhou 1291 ) and many others. 

Applying the Hamiltonian formalism to stochastic control problems, the class of Forward-Backward 
Stochastic Differential Equations (FBSDE in short) naturally arises in form of a partially-coupled 
system of the forward equation for the controlled diffusion and the adjoint backward equation for 
„generalized Lagrange multipliers". FBSDE with infinite (or random) time horizon are still subject 
to intensive study. The question which is quite delicate is the behaviour of the solution processes 
at infinity. There are several papers (e.g. Pardoux 11191 , Peng and Shi 1231 . Yin 1261 . Wu 1271 ) an- 
swering this question under different sets of assumptions both on the coefficients of the FBSDE 
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and on the terminal condition. Some of these papers naturally require that the terminal condition of 
the backward equation at infinity (in certain sense) is given in advance with suitable properties that 
consequently determine the space for the solution processes. Unfortunately, this is not directly appli- 
cable to stochastic control problems on infinite time horizon since the terminal condition is usually 
not known. Therefore, a different approach has to be introduced, namely we specify the solution 
space of random functions which determines asymptotical behaviour of the processes, suitable for 
our purpose. This approach appears already in Peng and Shi 0231 where however the conditions on 
coefficients, are rather restrictive - global Lipschitz property in all variables is assumed (which, in 
fact, excludes polynomial coefficients of higher order than two) and some special kind of so called 
weak monotonicity (also known as one-sided Lipschitz property). In this case, the solution process 
is vanishing at infinity and therefore, the terminal condition is zero a.s. In the paper by Wu Ii27l, a 
different monotonicity condition is assumed to obtain a solution process with non zero (in general) 
yet still a.s. constant terminal condition. The most general result is due to Yin |26| who weakens 
the assumptions to obtain the solution in some exponentially-weighted L'^ space for some suitable 
discount factor. Nevertheless, in the latter paper, existence of the solution to the backward part of the 
system employs the result by Pardoux f 191 based on the knowledge of the terminal condition. 

The novelty of present paper is twofold: First of all, the applicability of the maximum principle 
to infinite time horizon control problems has been studied under a natural set of conditions that are 
verifiable and may cover nonlinear problems with non-Lipschitz coefficients. Secondly, these general 
results have been used to solve the corresponding control problem for controlled stochastic logistic 
equation (which indeed involves non-Lipschitz terms), which is an important model of population 
dynamics. In this case also the solvability of the closed-loop FBSDE has been established. 
The paper is organized as follows: The discounted control problem is formulated in the second 
section. In the third section the existence and uniqueness theorem of the solution to FBSDE on 
infinite time horizon is provided. Fourth section contains the main result of the paper - the suffici- 
ent infinite time maximum principle for the discounted problem. In the fifth section, some standard 
examples are solved by using results of the fourth section to show the applicability. The last section 
which contains our main example is devoted to the above mentioned controlled stochastic logistic 
equation. 



2. Formulation of the problem 

2.1. Preliminaries. We are given a basic probability space (O,, T, P) with R''-valued standard Wi- 
ener process W = (W^t),>o. Let be the canonical filtration of W, i.e. = a{Ws; s < 
t), and be its P— null sets augmentation. We denote Toe ~ Vt>o-^* ^ -^^ Further, to 
simplify the notation, we write just 'a.s.' instead of 'P -a.s.'. We denote | • | and || • || the Euclidean 
norms in R" and R"^'' respectively. Further, •) stands for standard scalar product in R" and 
Tr(-) denotes the trace of a square matrix. 

Now, for /3 £ R and any Banach space X with norm || ■ ||a^, we introduce the space of random 
processes 

L^'' (R+; X) := < V : R+ x Q, ^ X : v is (-^t)t>o ~ progressive with 




(2.1) 



We denote (M+;X) just as L% (R+; X) and 



'^'t^oc C^+i 1^ • ^+ X Q, X : VIS (-^t)t>o — progressive with 

e^'ll^tlll-df < +0O, VT > o| . 



(2.2) 



2.2. Discounted control problem. The controlled state process {Xt)t>Q is a strong solution to the 
following controlled SDE in R" 
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dXt = b{Xt,ut)dt + a{Xt,Ut)dWt, Vt > 0, (2.3) 
Xo = X, 

where (7 is a convex subset of R''' , the mappings fe : R" x f/ R" and cr : R" x t/ -> R" satisfy 
conditions specified in the next section. 
Denote Had the set of all admissible controls as 

Uad^ {u:R+ xn^U -.ue L^j„^ (0, +oo; U)} . (2.4) 
Any process u(-) £ Uad is called an admissible control. The cost functional takes the form 

J(u(-))=e/ e-^'f{Xt,ut)dt, (2.5) 
Jo 

where / : R" x f7 — >■ R is the penalization (or appreciation) function such that J(u(-)) converges 
for every admissible control and /3 > is the discount factor. 
Further, define the value function v by 

V — sup J(ii(-)). (2.6) 

"(■)6W„d 

The goal is to find such a strategy u* (•) £ Uad so that the supremum in ( 12. 6t is attained in ii* (•)> 
w = J{u* (•)). For the discounted control problem ( 12.3) , (12.41 . ( 12.5) and ( 12.6b we use the abbreviation 
DCP. 

Define a generalized Hamiltonian function T-L associated to control problem (12.3) - ( 12.6) by T-L : 

R" X [/ X R" X R"^'* Ras 



'H{x,u,y,z) = {b{x,u),y) + Tr{a{x,uy z) + f{x,u) - I3{x,y). (2.7) 

The Hamiltonian is an analogue to the Lagrange function in the theory of constrained optimization 
since the variables y and z can be viewed as 'generalized Lagrange multipliers' and the functions 
b and a as the constraints for the dynamics of the space process Xt- The additional term ~P(x, 
comes up from the Lyapunov function of the FBSDE system, see Peng, Shi |23 1. 
We note that neither the extremal point of T-L w.r.t. u nor the concavity/convexity w.r.t. (x, u) do not 
depend on the last term —l3(^x,y) . 

Suppose further that H is differentiable in x (with the gradient denoted as VxT-L) and we consider 
the following BSDE 

- dYt = V:,H{Xt, ut, Yt, Zt)dt - ZtdWt, > 0, (2.8) 
which, in application to the DCP, reads 

- dYt = (v,6(Xt, ut)Yt + D^a(Xt,ut) ■ Zt + V^f(Xt,ut) - fSYt^dt - ZtdWt, > 0. 

We use the notation Dxg{x, u) ■ z = Yfi=i ^xcr\x, u)z^ £ R", for z £ R"^'', where cr' denotes 
the i-th column of the matrix a. 

3. Solution to controlled decoupled FBSDE on infinite time horizon 
In this section, we study the following controlled FBSDE associated to the DCP 



dXt = b{Xt,ut)dt + a{Xt,Ut)dWt, Wt>0 (3.1) 
Xo=x e R", 

-dYt = (v^b{Xt,Ut)Yt + D^a{Xt,Ut) ■ Zt + Va:.f{Xt,ut) - PYt)dt - ZtdWt, Vt > 0. 

Since the system ( 13.1) is partially-coupled, i.e. the forward part does not depend on the solution of 
the backward equation, we can solve the forward part separately on R-|- obtaining the process X. In 
the next step, plugging the solution X into the backward part, we find the solution processes {Y, Z). 
We start with results on infinite time horizon SDE which is given by 
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dXt = b{Xt,ut)dt + a(Xt,Ut)dWt, yt>0, (3.2) 
Xo = X, 

where f/ is a convex bounded subset of R'' and x G R" is a deterministic initial condition. We furtiier 
assume that the mappings 

fo ; R" X ;7 -^R", 
a : R" X U ^R"""", 

satisfy the following conditions. 

(HI): 6(., •) and cr(-, •) are continuous on R" x U. 

(H2): There exists /ii £ R such that 

{xi — X2, b(xi, u) — h{x2,u)) < iii\xi ~ X2\^ , (3.3) 

for any {xi,X2,u) e R" x R" x [7. 
(H3): (Lipschitz property) There exists a constant L > such that 

||o-(a:i, u) — a{x2, u)\\ < L\xi — X2\, (3.4) 

for any (xi,X2, -u) G R" x R" x U. 
The following choice of the solution space will allow to apply Theorem 1 bellow to sufficient maxi- 
mum principle. As we will see, the suitable solution space ensuring proper asymptotics is the space 
L^"" (R+; R") for some /3 > 0. The following theorem holds. 

Theorem 1. [Existence and uniqueness] Let assumptions (H1)-(H3) hold. Then for each control 
m(-) £ Uad and every initial condition x G R", SDE it3.2l ) admits a unique solution X = ) g 

L^"'' (R+;R") where /3 - 2^1 - 2L^ > 0. 

Proof: Uniqueness follows from Lemma[T]below. The proof of existence is similar to Theorem 2.2. 
in Friedman 1 10] and it will be only sketched. First note that by virtue of (H1)-(H3) we have that for 
all e > there are constants , K (K independent of e) such that 

(x, b{x, u)) < (^1 + e) l^l^ + Ke, 
\\a{x,u)\\ < L\x\ + K, (3.5) 
for all a; > 0, It G U. Now for all n G N define 

r„ = inf{t > : > n}, (3.6) 

where {X") is a local solution obtained for trimmed coefficients on a ball of radius n and for time 
t G [0, r„]. Due to (H1)-(H3) and[33]one obtains 



supE fe^^e^--^) \Xr^,j] +{(3- 2(mi + e) - 2L^) E H 

t>0 L J Jo 



< \xof + I (a". + A"') < +00. (3.7) 

A similar estimate can be easily obtained with the supremum inside the expectation. By using the 
standard consistency argument and Lyapunov techniques with Lyapunov function V{x) = 1 + \x\^ 
we conclude that there is a limit process {Xt)t>o in L^"'' (R+; R") with /? - 2^i - 2L^ > 0. □ 

Lemma 1. [A priori estimate for SDE] Let assumptions (H1)-(H3) holdandu{-) G Wad be arbitrary 
but fixed. Let X^ — X"^'^'^ and X"^ = X"'-'''^ he two solutions to \3.2\ with initial values x\ and 
X2 respectively, belonging to TJ^~^ (R+; R"). Then the following a priori estimate holds 



supE (e-^'\Xt\^) + (/3 - 2/ii - 2L^) E / e-'^'\Xs\^ds < \xi - X2|^ (3.8) 
t>0 ^ Jo 

for I3>2m+ 2L^ and Xt = Xl - Xl 
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Proof: First assume that /3 G R is arbitrary and fix some t > 0. At the end of the proof, the right 
value of /? will be specified. Using Ito formula to e~''*|Xt ^ on [0, t] one arrives at 

Jo 

^\xi-X2f+2 f e-^'{Xs,biXl,u,)-b{X],Us))ds 
Jo 

+ 2 / e-'''{X,,{a{Xl,Us)-a{Xlus))dWs) 
Jo 

+ re-^'\\a{Xl,Us) - a{Xlus)\fds. (3.9) 
Jo 

Now, taking E(-) and employing (H2)-(H3) we obtain 

E(^e~''*|lt|2) + (/3-2^ti -2i")E^ e-^'\Xsfds < \xi - Xif ■ (3.10) 
Then for g > 2m + 2L^. we get iJM. □ 

Remark 1. One could easily obtain a similar estimate to i3.8i but with the supremum inside the 
expectation. In that case, we have to consider /? > 2/ii + 2L^ (C^ + 1) where C is the constant from 
Burkholder-Davis-Gundy inequality. For purposes of deriving the sufficient maximum principle, the 
estimation \3.8^ is satisfactory. 

Now, we will be interested in the solution to the BSDE i3.lt . We assume that we already know 
the solution X = X"' ' £ L^"'' (R+;R") to SDE ((S^J for some fixed u(-) £ Uad assuming 
(H1)-(H3). The couple {X, u) will be the (fixed) input for the BSDE CT . 

We start the study of the backward equation by further specifying the coefficients b, a, f. We assume 
that 

b:R" X C/^R", 
a : R" X U ^R"'"'', 
f -.R" xUx ^R, 

such that 

(H4): b{x, u),a{x, u) and f{x, it) are continuously differentiable in x for all u U. 

(H5): There exists fj,2 &R such that 

{yi - y2, V^6(x, u){yi - j/2)) < tJ-2\yi - V2\^, (3.11) 

for any (x, yi , 1/2) G R" x [/ x R" x R". 
(H6): There exists a constant M > such that 

d 

\\D^a{x,u)\\ := ^ 1 1 (a;, u)| | < M, (3.12) 

i = l 

for any {x,u) £R" x U. 

Using the transform Yt = e^^^Yt and Zt = e^^Zt, the infinite horizon BSDE i3.lt can be rewritten 
as 

~dYt = (v^b{Xt,Ut)Yt + D^a{Xt,ut)-Zt+e-^''Va^f{Xt,ut)yt^ZtdWt, Vt > 0, (3.13) 

If we show that the solution process {Y, Z) is in L^*^ (R+; R" x R"^'') then necessarily the origi- 
nal solution process (y, Z) has to be in L^"" (R+; R" x R"^'') since we have 

E/ e-^^\Ytfdt = E e"''*|e''*yt|^dt = E / e'^'\Yt\'^dt < +00, (3.14) 

Jo Jo Jo 

and similarly for the process Z. 



6 



The existence and uniqueness is given by Theorem 4 in Peng and Shi (23 1. In our setting, it is 
formulated as follows: 

Theorem 2. [Existence and uniqueness] Let assumptions (H4)-(H6) hold and (x^^ \ ^i')) be an 



admissible couple such that X, 



"(■) 



J+;R") and V^f{Xt,Ut) € L^"^ (R+;K"-) where 



2:13 



P >2fi2+ 2M^. Then BSDE I[3l3[ admits a unique solution (Y , Z) G Ljr 

Remark 2. The sign of [3 depends on the sign of the factor 2/i2 + 2M^. In case of ijL2 < one gets 
so called weak monotonicity condition as in Peng, Shi |23l allowing also < 0. As we will see later 
in Example 3, the controlled logistic equation naturally leads to fi2 > which implies /3 > 0. 

Using the previous theorem and (13.14) we conclude this section by 

Corollary 3. Let assumptions (H1)-(H6) hold and suppose that V xf{Xt,Ut) £ L^"'' (R+; R"). 

;+;R" X R"^'')/or/3 > 2^i2+2M2. 



Then BSDE dj.il ) admits a unique solution {Y, Z) € Lj^ (R4 

Remark 3. The above theorems stand valid if the assumptions (H1)-(H6) are fulfilled for all x G 
G C R", G domain, provided Xt G G for all t > a.s. 

3.1. Stability of BSDE under approximations. In this subsection we legitimate the approximation 
the BSDE by an equation with terminal zero condition. As we have already mentioned when dealing 
with DCP, the terminal value ^ of the solution process Y is not known. We will show that under 
(H1)-(H6) and natural assumptions on ^ (finite second moment) the approximation is stable, i.e. one 
can choose zero terminal condition or Tt - measurable its projections to obtain the same solution 
process. The following lemma connects the approach in Darling and Pardoux 1 20 1 and in Peng and 
Shi (23). 

Lemma 2. [Stability under approximations] Let (H1)-(H6) hold and let ^ be an K" -valued Too 
- measurable random variable with E|^p < +00. Further, for each n G N, let us consider two 
BSDE's 



and 



-dYt" = V,H(Xt,ut, Yt", Z]^)dt - Zl'dWt, t G [0, n) 

y" = 



-dYt" = VxHiXt,ut, Yt", Zl')dt - Z:^dWt, t G [0, n) 

y„" = := E [C|-F„] , 

where Vi'H(x, u, y, z) = Wxh{x, u)y + Dxo{x, u) ■ z + Vxfix, u) - (iy. 

We lay Y^ = and = := E [C|-Ft] , > n. 

Then 



(3.15) 



(3.16) 



lim 

n — S- + 00 



supE(e-^'iyr-y/f 



r+00 _ 

E/ i[o,„j(f)e-'^*|yr-y"i'dt 

Jo 



+E I lio,,,]it)e-^'\\Zr -ZT\fdt 
lo 



0, (3.17) 



for /3 > 2^i2 + 2Ar > 0. 
Proof: First observe that, using Jensen's inequality and integrability of 5, one gets 



E 



(e-^"|Cnf) =e-'^"E(|E(C|J-„)n < E (e"""!^!') 0. (3.18) 



Now, applying Ito formula to e '^^ \Y" — Y"\ one arrives at 



Ele-'^'lYt" ~Yt"f) +/3E 



-""ly" -y"Pds + E 



'^'\\Z" -Z"\\^ds = -Ele 



-/3n 



16.1') 



2E 



{\/xbiXs,u,){Yr - i>"),n" - YD + {DMXs,u,) ■ {z: - z:),y: - Y:)\ds. 

(3.19) 
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Now, due to (H5) and (H6), we arrive at 



E 



(e'^^ICnl') , VO<f <n. (3.20) 



+ iE/ e- 



Taking the supo<j<^ on both sides of d3.20t . we obtain 

sup E (e-^'\Yr - Yrf) + (13- 2^*2 - 2M")E [" e"""]?," - Y^fds 

0<t<n ^ ' Jo 



(3.21) 



To obtain the estimate for all t > and for /3 > 2^2 + 2M^ we write 

supE (e-"'!?," - i>"l') < sup E (e-^'lF," - y,"|') +supE (e-"'!?," - F,"!') 

t>0 ^ ' 0<t<n ^ ' t>n ^ ' 

<Efe-''"jC„|') +supE('e-'"|eth ^ 0. (3.22) 



The last term is due to definition of and Y" for t > and due to J3.18t . The convergence of the 
integral terms in (13.17) follows easily. □ 

Remark 4. In the above Lemma^ we could impose even more general conditions on ^ ensuring 
E (e-''"|,t„|2) ->Oasn->+oo. 

4. Sufficient stochastic maximum principle for the DCP 

In this section, we return to the discounted control problem and prove the corxesponding suffici- 
ent maximum principle. First, we employ the approach using the results from the previous section. 
Second, we apply results from 0ksendal et al. |12 | to our definition of Hamiltonian and prove the 
associated DCP using transversality condition. Finally, the connection between the two methods is 
shown. 

Theorem 4. [Sufficient stochastic maximum principle] Let (HI ) - (H6) hold and P > max {2/ii + 2L^2^2 + 2M2}. 
Assume moreover thatV^f(Xt,ut) G L^"-^ (R+;R")/or any admissible couple (X, u)from The- 
orem\l\ Further, let G( ) £ Wad d'ld X be the associated controlled diffusion process. Let us suppose 
that there exists a solution (Y , Z) to the associated BSDE \3.U such that 

(1) ■H(Xt,ut,Yt,Zt) = maxn(Xt,u,Yt,Zt), P ® dt - a.e., 

(2) (x, u) — > ^(x, u, Yt, Zt) is a concave function, P ® — a.e. 

Then u(-) — u* (■), i.e. u(-) is the optimal control strategy to the DCP. 

Proof: Take an arbitrary u(-) G Uad and examine the difference J(u(-)) — J(u(-)). The goal is to 
show that this quantity is nonnegative. Using the definition of J(u(-)) and Ti we have 

r+°° / ^ \ 
J(u(-)) - J(u(-)) = Ey^ e-^' [.f(Xt,ut) - f(Xt,ut)) dt 

= eJ^ e~^' {n(Xt,ut,Yt,Zt)-n(Xt,ut,Yt,Zt) + {b(Xt,ut) - b(Xt,ut),Yt) 

+Tr S^(a'(Xt,Ut) - a'(Xt,Ut))Zt} + P{Xt - Xt,Yt)) dt. (4.1) 
Clearly, the r.h.s. of ( I4.lt is finite hence we obtain 

E / Ttdt = lim E / Xtdt, (4.2) 
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where It is the integrand of ( I4.lt . 

Further, since (H1)-(H6) hold we know by Corollary |3] that each of the three solution processes 
X, X, Y belongs to L^^^ R") and therefore there exists a sequence (T'n)„gi>i , Tn +00 as 
n — >■ +CX3 so that 



E 



(4.3) 

where we have applied Cauchy-Schwarz inequality and the Young inequality 2ab < ^a^ + ife^, 
a, fo e R. By ( l42t and K3[ we have that 

■ rT„ 

J{u{-)) - J{u{-)) = lim E / Itdt + e-'^'^"{XT^~XT^,YTj . (4.4) 
Now, applying Ito formula to the last term in the bracket and taking E(-) we arrive to 

E [e-''^"(XT,. -Xt„,Yt„)] ^eJ e-"' (^{h{Xt,ut) - b{Xt,ut),Yt) 
+Tr { {a'{Xt, ut) - a'{Xt, ut))Zt} - - Xt, Y) 

~{Xt - Xt,V:,niXt,ut,Yt,Zt))) dt. (4.5) 
In view of the equality ( 14.4b . we finally arrive at 

J{u{-)) - J{u{-)) ^ lim E f "e^^'(n{Xt,ut,Yt,Zt)~n{Xt,ut,Y,Zt) 

-{Xt - Xt,\7^'H{Xt,ut,Yt,Zt))) dt. (4.6) 

By the concavity of H in {x,u), we have that 

H{Xt,ut,Yt,Zt) -H{Xt,ut,Yt,Zt) - {Xt - Xt,\7^n{Xt,ut,Yt,Zt)) > 0. (4.7) 
Therefore, we deduce that 

J(S(.)) - J(w(.)) > 0, Vu(-)€W„<i, 
which proves that u(-) is indeed the optimal control. □ 

Now, we provide a similar version of sufficient maximum principle proved in P121, Theorem 4. 1 using 
the so called trans versality condition (TVC). The fact which may be inconvenient for the controller 
is that the TVC has to be verified (theoretically) for every admissible control u(-). Moreover, this 
criterion, in general, cannot be explicitly verified in terms of the coefficients b, a, f and /3 of the 
DCP. On the other hand, TVC has an economical interpretation for the DCP, namely it expresses the 
additional benefit of one unit of good for time increasing to infinity. 

Theorem 5. [Sufficient stochastic maximum principle] Let u(-) £ Had and X be the associated 
controlled diffusion process. Let us suppose that there exists a solution (Y , Z) to the associated 
BSDE nil such that 



(1) H{Xt,ut,Yt,Zt) ^maxn{Xt,u,Yt,Zt), P ® dt - a.e., 

(2) {x, u) — > H{x, u, Yt, Zt) is a concave function, 'P ® dt — a.e., 

(3) the transversality condition (TVC) 



lim Efe-^VXt-Xt.yt)) <0, (4.8) 



holds for every X = X"<',ii(-)e Uad- 
Then u{-) — u*(-), i.e. u(-) is the optimal control .strategy to the DCP. 
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Remark 5. [Sufficient condition for TVC] We will show that the conditions (H1)-(H6) immediately 
imply the TVC, which provides an easy-to-verify approach for the controller 

Indeed, we have shown that under (H1)-(H6), the solution processes to the FBSDE (IJ.il > 

(X, Y, Z) = ^^,u(-).x^ 

{X Y Z) = (X^'"''' Y"^'"('''"*^ ^x,u(-),x-^ 
are both in (R+;R" X R" X R"^'')/or/3 > max {2^ii + 2L^ 2^2 + 2^/^} . 

Therefore, there exists a sequence , Tn +oo as n — )■ +oo .so that 

|E(e-'^^"(XT„ -XT„,yT„))| < iE(e-''^"|XT„ -XTj') + iE(e-^'^"jyTj')^^^0, 
which implies the TVC 114.81 . 

Remark 6. All results of section 4 holds also for time dependent coefficients h : 9.+ x R" x ?7 — > 

R", cr : R+ X R" X (7 R"'''', / : R+ X R" X f/ R under natural assumptions. 

Remark 7. [ Hamiltonian system ] Throughout the paper we assume that the Hamiltonian takes the 
form \2. 71 ). Nevertheless, one could easily verify that Theorem |4] holds also with the Hamiltonian 
function H replaced by a different Hamiltonian function H defined as 

H{x, u, y, z) = {b{x, u),y) + Tr{a{x, u)' z) + f{x, u) = Hix, u, y, z) + I3{x, y) (4.9) 

which enables us to rewrite the Forward-Backward Hamiltonian system as 

dXt = VyH{Xt,ut, Yt, Zt)dt + V,H{Xt, ut, Yt, Zt)dWt, Vt > a.s. 
Xo = x e R", 

-dYt= (y^H{Xt,ut,Yt,Zt) - - ZtdWt, yt>0, a.s. 

(4.10) 

Again, there is a correction term one has take into consideration, namely —jiYt in the driver of the 
BSDE. 

5. Examples 

In this section we provide two illustrative examples of discounted control problems with well 
known solution. It will be shown that the approach using our maximum principle lead to the solution 
as well. 

5.1. Example 1 - Production planning problem. The setting of the problem is taken from Bor- 
kar |6] and it is in fact the classical LQ problem whose solution is well known from e.g. dynamic 
programming, see 0ksendal II15I , chapter 11. In the feedback Markov setting, the optimal control is 
obtained in the form of the observed (Markov) diffusion state process Xt plugged into the solution 
of the associated Riccati ODE. 

Let us consider a factory producing a single item. Let X{-) denote its inventory level as a function of 
time and u(-) > denote the production rate. 77 denotes the constant demand rate and xi, ui resp. 
the factory-optimal inventoiy level and production rate. 
The inventory process is modeled as the controlled diffusion 

dXt = {ut -v)dt + adWt, yt > 0, (5.1) 
where cr is a constant. The aim is to minimize over non-anticipative u(-) the discounted cost 

J(u(-)) = Ey e-^'(c{ut~uif + h{Xt~xifjdt, (5.2) 

where c,h > are known coefficients for the production cost and the inventory holding cost, resp. 
and /? > is a discount factor. 
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The minimization task is converted to maximization by taking — J(ii(-)) as the functional. 

Firstly, it is easy to see that (H1)-(H6) hold for ni = fi2 ~ L — M — and therefore the DCP is 

meaningful for all /? > 0. The Hamiltonian of this control problem is 

H^x, u, y, z) = {u — ri)y + az — c(u — iii)^ — h(x — xi)^ — l3xy 
which is a concave function in (a;, u). The driver of the backward adjoint equation is obtained as the 
derivative of "H w.r.t x, i.e. 

d 

h{x, u, y, z) = ^^(^. y. ^) = -2/i(a; - xi) - Py 
and the associated BSDE is 

- dYt = {-2h{Xt - xi) - PYt) dt - ZtdWt, > 0, a.s. (5.3) 
To find the extremal (maximal) point of T-L we lay 



d 

— u,y,z) = y - 2c{u - Ui) = 



ut^^+ui. (5.4) 



which leads to 



Further, assuming the solution to BSDE ( 15. 3t in the feedback form 

Yt = iftXt +ipt, > a.s. (5.5) 
for some deterministic functions Lp, ip in one can arrive to Riccati system of ODE's similarly as 
in (13. 

5.2. Example 2 - Optimal consumption rate. The problem of optimal consumption rate is taken 
from 0ksendal U2i where it is solved using a different definition of Hamiltonian. In our setting, we 
can, in fact, follow exactly the solution procedure step by step showing the same result. The main 
difference is that using our approach, one immediately knows whether the transversality condition 
holds just by examining the coefficients. 

Consider an agent whose wealth evolves according to the following controlled bilinear SDE in R 

dXt ^Xt{iJ.- ut)dt + aXtdWt, > 0, 

Xo = a;o > 0, (5.6) 

where u{-) is the consumption rate process, /i and a are some real constants (unlike in 11121 where 
they are assumed time dependent and bounded). The aim is to maximize over all strictly positive 
controls u(-) bounded by some constant A' > the discounted cost functional 

r + ao 

J{u{-)) = Ej^ e-'^' In [utXt)dt, (5.7) 

First we see that (H1)-(H6) hold for ni — fi2 ~ f^, M — L = \a\. Further, one has to verify that 
the following condition holds 

E/ e-'''\\V^f{Xt,ut)\fdt<+oo. (5.8) 
Jo 

To see it, realize that the solution to ( 15. 6t is a geometric Brownian motion 

Xt = XQ exp (^aWt - ^cT^t + ' (5-9) 

so the condition reads 

r + QO -t 

E/ e""'— -dt < +00. (5.10) 

Jo \^t\ 

Since we have 



u 



E 



1^. 



<Xo^exp{{3a^ -2fj, + K)t} <+oo, (5.11) 

Theorem|4]may be applied for all /? > max {2^i + 2a^ , K + 3a^ - 2^}. 
The Hamiltonian of this control problem is 

nix, u, y, z) = x{fj, — u)y + xoz + ln(j:u) — j3xy (5.12) 

which is a concave function in {x,u). The driver of the backward adjoint equation is obtained as the 
derivative of H w.r.t. x, i.e. 

d 1 
j-'Hix, u,y,z) = (fi - u - /3)y + az + -. 
ox X 

To find the maximal point of H we lay 



which leads to 



and the associated BSDE is 



-^Hix, u, y, z) = -xy + - = 0, 
OU u 



ut = (5.13) 

XtYt 



dYt^ {{fi^nt- l3)Yt +(jZt + -L]dt~ ZtdWt, > 0. (5.14) 



The solution to BSDE J5.14t for a general admissible control u{ ) can be find along similar lines as 
in L12J . We obtain 

Y. = ^. (5.15) 



Finally, using l l5.13t and l l5.15t the optimal control is 

ut^P. (5.16) 

6. Controlled Stochastic logistic equation 

In this section we assume that the controlled dynamics is given by a stochastic differential equation 
with controlled logistic equation as a special case. It provides an example of a system with con- 
cave nonlinear coefficients. Here, the DCP is defined as follows. A one dimensional controlled state 
dynamics is given by 

dXt ^ XtF{Xt,Ut)dt + a{Xt)dWt, > 0, 
Xo = xo, (6.1) 

where it(-) is a control process with values inU = [U,U]x [V, \/] C M+ = [0, +oo)^ and a;o > 
is a deterministic initial value. The functions F : M. x U — >■ R and a : R — J- R are continuously 
differentiable in x,F is continuous in (a:, u), functions b{x, u) = xF{x, u) and u{x) are concave in 
all their variables and we further assume that 

• cr(O) = 0, 

• (t(-) is globally Lipschitz, i.e. there exists L > constant such that 

\cr(xi) — o-{x2)\ < L\xi — X2I, V(a;i,a;2) £ R^, 

• there exists M > constant such that \a' {x)\ < M,\tx £ R, 
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• there exists /ii G R such that 

(Xi — X2) [xiF{xi, u) — X2F(X2, u)) < lll\xi — X2f , yxi, X2 > 0,u £ U, 

• there exists /i2 G R such that 

{yi -^2)^ (^F{x,u) + x-^F{x,u)^ < fi2\yi -yif, 

Vx>0,u€ U,{yi,y2) G 

• there exist constants r, R,C > such that 



-F{x,u) < C, Va; G {0,r),u G U, 
F{x,u) <C,yx £ {R,+oo),u eU. (6.2) 

Notice that the two functions b{x,u) = xF{x,u) and o-{x) fulfil the assumptions (HI) - (H6). 
Assume further that 

xF {x, {U,V)) < xF{x,u) < xF {x, , Va: > 0, Vft G U, (6.3) 

and denote the corresponding (constant) controls as uiower ~ {U_j V) and Uupper ~ {U,V^. 

Note that the notation F{-,-) in the drift in equation ( 16. Il l covers situations of controlled drift such 
as: xF(x, u) = ax{u\ — bu2x) for u = (wi, M2)', in which one can recognize the logistic equation 
with controlled both the saturation level of the diffusion and the speed of reaching this level. 

The set of admissible controls Uad is defined as 

Uad = {u:m.+ Xn^U -.ue L^jr^i,, (0, +00; U)} . (6.4) 

The functional to be minimized over all admissible controls ii(-) is 

J(u(-)) = e/ e-'^' [cX^ + h\utf) dt, (6.5) 
Jo 

where c,h>0 and /3 > is a discount factor. 

First we start by showing that the solution to ( 16.1) for xq > stays in (0, +00) for alH > a.s., i.e. 
it does not explode in finite time and the point is not attainable for the solution a.s. 
We prove it in two steps. First, we prove this statement for the solution X" = X to ( I6.lt computed 
for an arbitrary but fixed constant control S G (7. To proceed define for every n G N and S > two 
stopping times 

r„ = inf{t > : |Xt| > n}, 

Ts^ud{t>0:\Xt\<S}. (6.6) 

Note that by a.s. continuity of X, t„ is a nondecreasing sequence of times almost surely and ts is a 
nondecreasing with 5 decreasing to 0. Therefore there exist stopping times e (explosion time) and k 
(hitting time of 0) so that 

T„ e, n — ^ +C)0, a.s. 

TS /• K, 5^0+, a.s. (6.7) 

Further, the infinitesimal generator of the Markov semigroup corresponding to X is given by the 
formula 



£f{x) = xF{x,u)^f{x) + L^\x)^i{x), 
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for / £ C^(R+). Using the Lyapunov function V{x) = 1 + ^+2:^ and assumptions on ct(-) and 
F{-, ■) one gets 



£V{x) = xF{x,u) {--^ + 2x]+ ^a'^ix) ( 4 + 2 ) < Kil + - + a;') = KV{x), 



X 

where K = max {L^ + C, + 2/ii } for a; G (0, r) and if = max {L^, 2C + L^} for x > J?. 

Further, V{x) is radially unbounded, i.e. 

qR = inf = 1 + i?^ — >■ +oo as i? — > +oo, 



1 2 

= inf V(x) — 1 + - + S +00 as 5 ^> 0+. 



(6.8) 



Now applying Ito formula to V (^XtAT„ATs^ and taking E(-) 



one amves at 



V(xo) > E [l 



V XtAr, 



E 



- E 



> E 



+ E 



■'-{TnATa<t}l{T5<T„} 

>r{Ts < t)qs+r{rn < t)q„. 
Therefore, employing ( 16. 8t we have for each t > fixed 



(6.9) 



" {rn <t)< 
" (rs <t)< 



Vixo) 



e^V(xo) 
qs 



0. 



(6.10) 



Taking into account l l6.7l l and ( 16.10b we arrive at 

ma.x{r{e<t),r(K<t)} = 0, \/t>0. (6.11) 
Sending t +cxd, we finally obtain 

max {P (e < +oo) +oo)} = 0, (6.12) 

which means that with probability 1 the solution X to equation l l6.1b computed for some fixed con- 
stant control u £ U and for xq > does not explode and does not attain in finite time. 

In the second step we show that for any admissible control u(-) G Uad, the solution X — X"' ' to 
equation J6.1I > satisfies 

X"'™" < X"''^ < X"""", > 0, (6.13) 
for the common initial condition xq > 0. 

The result of the previous step holds in particular for X"'™" and For proving the unattaina 

bility of 0, a comparison theorem will be the key tool. We employ Theorem 1.1 and Remark 1.1 ii 
Ikeda, Watanabe, L13J for special choice of functions 

/3i(t,a;) = 6i(t,X"'°"") = &2(i,X"'°"") = X"'°""F (X"'""", itiower) , 
/32(t,a;) = Xr'-'F(xr<-\^(f)). 
Using ( I6.3t we know that 



m 
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A t's)F A t's), liiowe,-) < A t's)F (x^^-\t A r^), A r's), ) (6.14) 

for all t > Q, xo > Q and = inf{t > : < 5}, 5 > 0. Applying Theorem 1.1 and 

Remark 1.1 in 1131 one gets 

< X"'"™(s) < X''<-^(s),Vs e [0,ra),V(5 > 0. (6.15) 

From the construction of stopping times Tg , ts (the latter now defined in terms of the process X"'"""' )> 
by ( 16.151 1 and from the first step of the proof it follows that 

< Ti < t's, (6.16) 

for all 5 > 0. Since ts —5- +oo almost surely as 5 — >■ 0+ we conclude that X"''^ does not attain 
for alH > almost surely. Nonexplosion of X"' ' can be proved similarly. 

By Remark[3]we conclude that Theorem|4](or Theorem|5) can be applied for the above DCR 



6.1. The controlled logistic equation. In the rest of the section we derive the form of the associated 
FBSDE and the Hamiltonian for a special choice of ■), namely for 3;_F(a;, = ax{l~bx)+'^u, 
for a, 6 > 0, M G [ui, 112], iii > 0, 7 £ R, i.e. the additive control is one dimensional process. The 
diffusion term is given by (j{x) = ax for some cr > 0. The one dimensional controlled diffusion 
therefore evolves according to the SDE 

dXt = aXt (1 - bXt) dt + -futdt + aXtdWt, 'it > 0, 
Xo = xo> 0. (6.17) 

The functional to be minimized over all admissible controls ii(-) is 

r + ca 

J(u(-))=E/ e-P^ {cXt +hut)dt, (6.18) 
Jo 

where c,h>0 and /? > is a discount factor Again, we will maximize — J(m(-)) and we see that 
the Hamiltonian is of the form 

'H(x, u, y, z) = a{x — bx^)y + yuy + axz — cx^ — hv? — fixy, (6.19) 

which is a concave function in (x, u). The driver of the backward adjoint equation is obtained as the 
derivative of w.rt. x, i.e. 

d 

— T-Lix, u, y, z) = [a — I3)y — 2abxy + az — 2cx. 
Therefore, the associated BSDE reads 

-dYt^[{a- l3)Yt-2abXtYt + aZt-2cXt]dt- ZtdWt, it > 0. (6.20) 
To find the maximal point of Ti w.rt. u we lay 

d 

— ?^(a;, u, y, z) = '^y ~ 2hu = 0, 
to obtain the argmax to the quadratic function in it given by T-L, i.e. 

7 

Umax = (6.21) 

Finally, the optimal control is obtained by studying the mutual position of Umax and the interval 
[mi, 112] which leads to the optimal control 

ut=u{Yt), (6.22) 



where 
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Ui 



2hJ/' y 6 



[-Ul,-U2\ 



(6.23) 



-U2. 



U2, y > 

It can be easily seen that u(-) is bounded by 112, continuous and nondecreasing. The last thing which 
is not obvious is that ut is indeed an admissible control, i.e. that FBSDE J6. 17t . i6.20t admits a 
unique solution with ut plugged. When this is shown then according to Theorem|4](or Theorem|5]l 
Ut is optimal. 

The corresponding FBSDE in the present case reads 



dXt = aXt (1 - bXt) dt + 7S (Yt) dt + aXtdWt, 
~dYt = [(a - l3)Yt - 2abXtYt + aZt - 2cXt] dt - ZtdWt, 
Xo = xo > 0. 



(6.24) 



FBSDE with the non-Lipschitz term —2abxy in the driver of the backward part are not covered (up 
to authors' knowledge) with the existing theory. 

We start with the local uniqueness theorem proved for the solution in the space L^r (R+;R'^) 
which is sufficient for finite time horizon problem. 

Lemma 3 (Local uniqueness). Let (^Xl, Y/, ^t)tg[tg to+s] ' * ~ 1, 2 foe two solution to FBSDE 
SK24[ for some 5 > 0, to > such that Xt^ = Xtg and Yf^^g = Yfl^g. Then 

{Xl,Y^,Zl) = Yt\Z^t) , e [toM + 5], P - a.s. 

Proof: Denoting Xt = X^ - Xl , Yt = Y? ~ F/, Zt = Z'l - Z\ , Vt € [to, to + 5], applying Ito 
formula and using standard estimates one obtains 

sup E\Xt\^ < C \E\Xt,f +^1^0+6^] exp ( KS + KE [ lY^^fds) , 



sup ElVil 

te[to,to+4] 



1 IE 

ei 



to+s 



IZsl'ds < C 



to+s _ 

E\Yto+sf+E \Xsfds 



X exp K5 + KE 



to+s 



IXtfdt 



(6.25) 



where C,K > are some constants and £1 > is sufficiently small. 

In a similar way one obtains the estimates with sup(-) inside the expectation 



sup \Xt 

telto,to+s] 



C [E\Xtof + E\Yto+sf] exp (^KS + KE \Y^ 



ds 
(6.26) 



(l-2Ci£3)E 



sup I Ft I 

te[to,to+«] 



El £3 



E 



to+s 



+CE 



to+s 



in'l' ■ \Xs\'ds, 



ZsYds < C \E\XtoV +E\Yto+s\ 



(6.27) 



where again C,K > Q are some constants, Ci > is the constant from Burkholder-Davis-Gundy 
inequality and ei, £2, £3 > are some sufficiently small numbers. 

Since Xto = Yto+s ~ uniqueness of the process X follows from ( I6.26t . Uniqueness of the 
processes Y, Z then follows from ( I6.27t . □ 



16 



To sketch the construction of the solution we first introduce for each n £ N the approximated 
equation 

dXt = aX" (1 - bX") dt + ju {YD dt + aX"dWt, 

-dVr = [{a - l3)Yr - 2ab{X^ A n)Ft" + aZl' - 2c{X^ A n)] dt - Zl'dWt, 

Xo" = a;o > 0. (6.28) 

The second equation in the system J6.28l > has Lipschitzian r.h.s. and it is well known (Yin, 1261 ) 
that it admits a unique solution {Xt,Y", Z")^^^^ in L^"*^ (K+; R^) for each n G N and for some 
exponential weight /3 > specified later. We also note that the approximate drivers hn{x,y, z) = 
(o — I3)y — 2ab{x A n)y + az — 2c{x A n) converge pointwise to the original driver h{x, y, z) = 
{a — P)y — 2abxy + az — 2cx as ti — >■ +cxd and that the solution process X inherits all the properties 
(nonexplosion etc.) proved above using the Lyapunov function. 

Using standard techniques it is not difficult to derive the following uniform estimates (in n) of the 
process X for P > + 2a + + max {7112 , ^ } 

supE [e^^'lJCri^j +(l-ClCr£3)E 
t>o L J 

supE |'e-'"|Xi|^] +(l-Cicr£3)E 

where e, £3 > are some sufficiently small numbers and X = X™ — X", Y = — Y" for each 
m,n € N. 

Now, let n,N € N be arbitrary and t > 0. Using previous estimates and Chebyshev inequality 
one can examine the boundedness of the process X in probability (uniformly in t on any finite time 
interval [0, T] for each T > fixed) in the following meaning 

PdXri > iV) =p(e-.'|Xr| > e-.'iv) < ^ ^J,^' ^ < ^^^I^^^O' (^.31) 

which will be a comer stone for the construction of the solution on finite time horizon. Therefore we 
consider the finite time horizon approximated equation 

= aX^ (1 - bX^) dt + 75 {Yt") dt + aX]^dWt, t G (0, T] 
~dYr = [{a - P)Yt" - 2ab{Xl' A n)Yt" + aZ^" - 2c(Xr A n)] dt - Z^^dWt, t G [0, T) 
Xo" = a;o > 0, 

K?^ = 0. (6.32) 

Employing classical results on BSDE we know that there is a unique the solution to i6.32t in 
L^"" ([0, T] ; R^) for each T > fixed and for each n G N. 

Further, let Fat be a cylindrical subset of R"^ of the form 

Fjv = B(0,iV) X R X R, 
where -6(0, TV) is a ball of radius X G N with center in origin. Now we are ready to prove the 
consistency theorem. 

Theorem 6. [Consistency] Let T > be arbitrary but fixed, N £ be some (large enough) index 
andm,p G N, m,p > N.Let/S > a'^ + 2a+ ^ +ma.x {'yu2, ^} and let {XJ" ,Yt"' , ZD^^ic^t] 
and {Xf, Yf, Zf )^^^^ j,j respectively be two solutions to FBSDE li6J2[ in L^"" ([0, T]; R^) com- 
puted for n = m, n = p respectively. Then 

(Xr, Yr, ZD (oj) = {Xf, Yr, Zf) (c.), Vt G [O, T], G nr„ , (6.33) 



supe ^ \Xt 

t>o 



^ 2 7'^2 

< a;o + — < +00, 



(6.29) 



-/3t| O |2 

supe ^ I At I 

t>0 



\Y,\'ds, (6.30) 
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where r!r„ = {c. £ : {XT, , ZD {uj) G Tn, {Xf , Yf, Zf) (lj) G Tn, Vt G [0, T]}. 
In other words, all the trajectories which stay in the cylinder Fjv coincide. 

Proof: Let cj G fir^ and denote 1/" = (Xf, V", for all n G N. Then for (V;'') (cj) and 
(Vt™) (a;), m> p,we obtain 

F/(a;) = /I, (yf (c)) Ir, (Ff (a;)) + £ h, (\//(c.)) Ir^ (1/^(0;)) ds - (^j^ Z^^dW^j (c), 



(6.34) 



Fr(c^) = (K"(^)) lr,„ + C hm (K™(c^)) Ir^ ds 



^ ZTdWs^ (lo). 
(6.35) 

We also notice that 

hp{x,y,z)lTp{x,y,z) = hm{x , y , z)1t j,{x , y , z) = h{x,y,z)lrp{x,y,z) and 
2)lr„(2;,y,z) = hm{x,y, z)lr„{x,y, z) = /ijv(a;, y, 2)lr„ (x, y, z). 

(6.36) 

Since Fjv C Fp C F^ for m > p > N, (Vf) (lj) also satisfies the same integral relation (16.35) and 
the forward equation of the system, i.e. (V/) (u) solves l l6.35t as well, and by the local uniqueness 
(Lemma|3]with n = m) it follows that (V/) (a;) = (Vj") {(-j), Vt G [0, T]. 

Further, we can prove along similar lines that (Vf) (oj), (Vt'") (uj) solve l l6.32t for n — N. Putting 
both arguments together we see that all the solution trajectories that stay in some cylinder (for some 
index) solve also all the equation with larger index and all such trajectories coincide. □ 

Proposition 1 (Construction of the solution). Under the assumptions of Theorem |6] there exists a 
unique solution {Xt, Yt, Zt)^Q j,^ G L^"'' (R+; R'') of the finite horizon FBSDE 

dXt = aXt (1 - bXt) dt + 7M {Yt) dt + oXtdWt, 
~dYt = [(a - l3)Yt - 2abXtYt + aZt - 2cXt] dt - ZtdWt, 
Xo = xo> 0, 

Yt = 0. (6.37) 
Proof. Keeping the notation from the previous proof notice that 

f2rjv = ^p,N n Q,m,N, 

where n„,iv = {a; G ; (Xt", Ft", Zf) (oj) G Fjv, G [0, T]}, forn G N and due to estimate 
( I6.31t we have that P (f7p,jv) — > 1 uniformly in p. Thus we put for any fixed p > N 

iV— > + oo 

n = [J Qp^N, withP(^f2j = 1. (6.38) 

jveN 

The unique limit solution process to FBSDE 

dXt = aXt (1 - bXt) dt + -yu (Yt) dt + aXtdWt, t G (0, T] 
^dYt = [(a - l3)Yt - 2abXtYt + aZt - 2cXt] dt ^ ZtdWt, t G [0, T) 
Xo = a;o > 0, 

Yt = 0. (6.39) 
is obtained for all i G [0, T] as 
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for each N £ N. We stress that this construction is independent of the choice of p due to the consis- 
tency theorem. □ 



Construction of the solution to the original FBSDE J6.24t now follows the lines of analogous proof 
in Pardoux II20I with the process (Xt, Yt, Zt) from Proposition [T] as its finite time horizon approxi- 
mation. That is, the solution to J6.24b is obtained as a limit of solutions {X", Y", Z")^^g where 



dXt = aX" (1 - bX") dt + 75 {YD dt + aX^dWt, t G (0, n] 
-dYt" = [(a - li)Yr - 2abX^Yt" + aZ^ ~ 2cXr] dt - Z^^dWu t G [0, n) 
Xq = a;o > 0, 

= X",t > n, 
Yt" ^0,t>n 

Zt^O,t>n. (6.41) 
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